Evidence on “What Works”: An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs

The What Works Clearinghouse (WWC) was recently charged by the Institute for Education Sciences 1(IES) with collecting, appraising, and reporting on the “strength and nature of scientific evidence on the effectiveness of education programs, products, and practices”(see http://www.ed. gov/offices/OERI/whatworks/). In response, the WWC developed and recently released a set of standards for selecting empirical studies yielding research-based evidence on effective educational programs (Valentine & Cooper, 2003). Consistent with the language and references to scientifically based research in the No Child Left Behind Act of 2001, the language in the WWC standards also refers to testing of hypotheses using experimental designs, with a preference for randomized field trials for judging effectiveness of educational initiatives (Eisenhart & Towne, 2003). The larger mission of the WWC appears to follow in the traditions set by the Cochrane Collaboration in medicine and the Campbell Collaboration in education, where the aim is to collect and synthesize results from a series of topically-related experimental studies with meta-analyses, permitting more dependable inferences to be drawn about particular interventions than is possible with any single study. The WWC standards are formulated as four “general”, eight “composite”, and a number of more specific questions on a screening instrument, titled the Study Design and ImplementaEvidence on “What Works”: An Argument for Extended-Term Mixed-Method (ETMM) Evaluation Designs